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[57] ABSTRACT 

An artificial network for encoding the binary on-state 
of one-out-of-N inputs, say j, when only one state is on 
at a time wherein the jth on-state is represented by a 
suitable output level of an N-input MP type neuron 
operating in the non-saturated region of the neuron 
output nonlinearity. A single line transmits the encoded 
amplitude level signal to a decoder having N single 
input neural networks. The N outputs of the decoder 
are in the off-state except for the output corresponding 
to the active input node of the encoder 

12 Claims, 5 Drawing Sheets 
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Prior Art 




FIGURE 2 
Prior Art 
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APPARATUS FOR A NEURAL NETWORK 
ONE-OUT-OF-N ENCODER/DECODER 

FIELD OF INVENTION 3 
The invention pertains to the field of artificial neu- 
rons and neural networks and more specifically to the 
use of artificial neural techniques for the implementa- 
tion of a remotely operable oneout-of N selector means 
or signalling device. 1° 

BACKGROUND TO THE INVENTION 

Artificial neuron technology is emerging as a viable 
design alternative for many applications including com- 
munications and control. As practical commercial com- * 5 
ponents become increasingly available, solutions to 
signalling and control problems utilizing these compo- 
nents will be needed. 

Remote signalling and/or switching has basic utility 
in a broad spectrum of application. The current inven- 20 
tion is for a novel system using an elementary form of 
the McCuHough-Pitts (MP) neuron for encoding a se- 
lector switch position for transmission to a remote loca- 
tion where the received message is decoded so that the 
same selector position is assumed at the receiver site. 25 
The decoder at the receiver is another artificial neural 
network comprised of known or standard neural net- 
work components. 

In an article entitled "How to Solve the N Bit En- 
coder Problem with Just Two Hidden Units" by Leonid 30 
Kruglyak in Neural Computation 2, 399-401 (1990), 
Massachusetts Institute of Technology, Kruglyak im- 
plies that a three layer network of the standard feedfor- 
ward architecture can solve the N input-N output en- 
coder problem with just two hidden units. This struc- 35 
ture is i sometimes referred to as a N-2-N encoder/de- 
coder network. Other known prior art required a N- 
logaN-N network with hidden Log2N units to solve the 
problem, as reported in "Parallel Distributed Process- 
ing: Explorations in the Microstructures of Cognition", 40 
Rumelhart, D. E. and McClelland, J. L. t eds., Vol. 1, 
Chap. 8, p. 318, MIT Press, Cambridge, Mass. 

FIG, 1 shows the structure implied by Kruglyak 
consisting of encoder unit 10 and decoder unit 20, The 
encoder unit 10 consists of a N-position slider switch 45 
with wiper arm 11 and contacts 13. Wiper arm 11 is 
connected to a constant voltage source 9 which may, at 
any time, be applied to only one contact 13 out of a total 
of N contacts. Each contact 13 is connected to neural 
processing elements (PE) 17 and 19 by synaptic 50 
weighting elements 15. Signals on output lines 14 from 
PE 17 and 19, represents the encoded switch position (1 
through N) of wiper arm 11. 

FIO. 2 shows a typical MP neuron and its relation- 
ship to the encoder structure 10 of FIG. 1. The inputs 55 
(1 — N) are applied to input nodes 13 that are each con- 
nected to a PE unit by synaptic weighting elements 15 
(Wl, W2, . . . ,WN). Summing unit 12 adds these 
weighted inputs together with an appropriate threshold 
offset 0. The output of summing unit 12 is applied to an 60 
output (squashing) nonlinearity 16. Nonlinearity 16 
typically has a sigmoidal transfer characteristic. In this 
manner, PE elements 17 and 19 encode the state of 
selector switch arm 11 for transmission over output 
lines 14. 65 

Neuron decoder unit 20 of FIG. 1 consists of N PE 23 
units (PE1— N) each having two synaptic connections 
21, one to each line of the input line pair 14. The output 
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of the PE 23 units constitutes a N-terminal output array. 
Upon proper selection of encoder 10 and decoder 20 
synaptic weights, output terminals of unit 20 may all be 
active at various voltage levels. However, the output 
voltage of the PE unit, PE;, corresponding to the se- 
lected jth input active terminal 13, is maximum relative 
to the other N-l output terminals. 

Although the output of decoder network 20 contains 
all of the necessary information required to determine 
which one-out-of-N inputs was activated, it is not fully 
decoded in the sense that all output terminals 25 will 
have analog voltages that are representative of their 
"closeness" to the selected jth input terminal 13 of en- 
coder 10. Additional processing would be required in 
order to select the jth output terminal 25 having the 
largest output voltage. A winner-take-all (WTA) net- 
work which may be used for this purpose, accepts the N 
output terminals 25 as its input and activates the one- 
out-of-N of its output terminals corresponding to the 
input terminal having the TnnTmnim voltage value. 

Many possible implementations for the WTA net- 
work are known to those skilled in the art For example, 
the MAXNET neural network described in the book 
"Adaptive Pattern Recognition and Neural Networks" 
by Yoh-Han Pao, section 7.2, p. 173, Addison-Wesley 
Publishing Company, Inc., 3989. MAXNET uses lateral 
inhibition to accentuate and select one candidate node, 
among a set of nodes, that has the greatest output," It is 
described as a "feedforward Hamming net maximum • 
likelihood classifier of patterns of binary inputs, cor- 
rupted by noise." Although the input patterns in this 
application are analog voltages, the property of the 
network is to select the input with the largest magni- 
tude, treating each input as a possible one (on-state) and 
selecting the largest valued input as the most likely 
on-state candidate. 

Thus, prior art teaches that an N-state encoder/de- 
coder system such as shown in FIG. 1, requires that 
transmission line unit 14 be comprised of at least two 
distinct lines. In general, it would be advantageous to 
reduce the number of transmission channels required 
particularly if the reduction is achievable at no, or rela- 
tively low, cost. 

SUMMARY OF THE INVENTION 

An artificial network for encoding the binary on-state 
of oneout-of-N inputs, say j, when only one state is on 
at a time is described. This jth on-state is represented by 
a suitable output level of an N-input MP type neuron 
operating in the non-saturated region of the neuron 
output nonlinearity. A single line transmits the encoded 
amplitude level signal to a decoder comprising N single 
input neural networks. The N outputs of the decoder 
are in the off-state except for the output corresponding 
to the active input node of the encoder. 

Another object of this invention is to minimize trans- 
mission costs. By employing a single transmission line 
for sending the encoded message significant cost sav- 
ings may result in a signalling system requiring remote 
sensing of one on-state out of a multiplicity of inputs. 

A further object of this invention is to use neural 
network technology for its implementation, thereby 
facilitating the interface with other neural components. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The novel features believed characteristics of the 
invention are set forth in the appended claims. How- 
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ever, the invention itself, as well as other features and Decoder 200 receives the encoder output, vj, from 

advantages thereof, will be best understood by refer- element 114 and creates N distinct output values (zi-rd 

ence to the description which follows, read in conjunc- by means of N single input MP type neurons each com* 

tion with the accompanying drawings, wherein: prising a synaptic weighting element 221 and a PE 223. 

FIG. 1 illustrates a prior-art N-2-N encoder/decoder 5 Each weighting element of the set {W*} is distinct for 

neural network. k= 1, 2 N. The output of each synapse 221 with 

FIG. 2 illustrates a prior-art MP type artificial neuron weight W* is applied to a PE 223 element comprising a 

with N inputs, summing element with a corresponding offset value 0*. 

FIG. 3 illustrates the current preferred embodiment Thus, each synapse 221 and summer of clement 223 is 

of the present invention, a N-t-N encoder/decoder 10 equivalent to a single input MP type neuron operating 

neural network. in the non-saturating region of the output nonlinearity 

FIG. 4 is a graph showing decoder output character- (ONL 16 of FIG. 20). The kth output, z*> of decoder 

istics for exponentially weighted encoder synapses, 200 may, therefore, be expressed as 
linearly weighted decoder synapses and exponentially 

weighted decoder offsets. 15 x^v^-ryin+e* (3) 

FIG. 5 is a graph showing decoder output character- 
istics for linearly weighted encoder synapses, linearly In order for z* to be a suitable function for driving the 
weight decoder synapses and quad radically weighted WTA 30 network, it must have a maximum value at 
offsets. k=j, i.e., when the message vj is received by decoder 

FIG. 6 is a graph showing decoder output character- 20 200, the jth output terminal must have a value zj that is 

istics for linearly weighted encoder synapses, logarith- greater than all of the other N— 1 output values, or 
mically weighted decoder synapses and linear weighted 

decoder offsets. z >*k*j (4) 

DESCRIPTION OF THE PRESENT INVENTION 2 5 This requires that the first derivative of z* with respect 

The prior art as shown in FIG. 1 clearly implies that to k, z'jt, be equal to zero for k=j, or 
a N-2-N encoder/decoder with two "hidden" neurons, 

as processing elements 17 and 19 with their synaptic iv j kamJ = v^-y 1 « o (5) 
inputs 15 are sometimes referred to, is a minimum type 

configuration (Kruglyak, op cit). The current invention 30 Hence, 

results in a N-l-N encoder/decoder configuration 9 ^ _ 9>k 

wherein N input nodes are connected to a single MP y " ~ J 

type artificial neuron whose output provides an output Als0( ^ derivative Wfc » must ^ negativc m 

signal level representative of the input node selected. order t0 cnsurc a maximum rather ^ a or 
This single output signal is used by the decoder to acti- J3 

vate the one-out-of-N output lines that corresponds to t,V.'**/<-0"*~/ (fi) 
the selected input node. 

FIG. 3 representing the preferred embodiment is These two conditions are sufficient to ensure a maxi- 

comprised of two basic units: encoder 100 and decoder mum value at the proper output tenninal of decoder 

200. Also, an optional WTA network 30 is shown con- 40 200. However, depending upon the type of WTA 30 

nected to the N outputs of decoder 200 for uniquely network used, it may be necessary to impose a further 

selecting the one-out-of-N output terminals of decoder constraint. For example, the MAXNET network re- 

200 having the maximum value. quires that the maximum value, zj, also be positive, i.e., 

Encoder 100 has N input nodes 113, any single one of zj >0, or 

which may be in contact with selector contact 11. A 45 ' 

voltage from source 9 is applied through contact 11 to v^-p-^-y O) 
the selected node 11 3, say the jth node. The source 9 

voltage, e ft is applied to the jth synapse 115 character- These three conditions specified by expressions (5), 

ized by the scaling factor or weight, wj. Each of the N (6) and (7) above, can be satisfied by a large variety of 

input synapses 115 is applied to the summer 112 input. It 3U functions. For example, consider the encoder/decoder 

should be noted that the sub-structure in encoder 100 configuration shown in FIG. 3 when the decoder syn- 

compnsing weights 115 (Vi_^) and adder 112 consti- apt i c weights 115 are exponentially scaled uniform in- 

tute a MP type neuron of the type shown in FIG. 2 crement values, the decoder synaptic weights 221 are 

operating in the non-saturating, possibly linear region, linearly scaled uniform increment values, and the offsets 

of the output nonlinear element 16. Thus, except for the 2 18 are exponentially scaled uniform increment valuesk 

selector switch element 113, the encoder comprises a ^ follows: 
single MP type neuron. The output, y, of encoder 100, 
applied to transmission means 114 for coupling to the 
input of decoder 200, may be represented by 

60 

(1) 

If for convenience, e 0 =l, then 

y=yj (2) 65 



»y « 2/ for j = 1,2,. 
wjc = ok 

e k = -2* 

so that 



Thus, if each {vj} is unique, the value is representative i k = oVk . 

of the jth selected input node. 
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The first condition requires that the first derivative be 
equal to zero at k=j, or 

tkmi = aTfk - 2* In 2 ju;°0 

to that 

a»b2 

Also, because the second derivative is negative at k=j, 
or 

i"*.y=-V(/»2)*<0 



A further example obtains when vi is linear with 
respect to increments in index j, w* is logarithmically 
scaled with respect to index k, and 0* is linear with 
respect to index k, as follows: 



10 



the second condition is also satisfied. 

FIG, 4 is a plot of zjt versus k with j= 1, 2, 3, 4 as a 
parameter for a= In2=*0.7. It may be seen that the maxi- 
mum value occurs when k=j. 

Because the lower curve (z*, j= 1) is always negative, 
it may be desirable to impose the third condition requir- 
ing 

= atik + 9 k J v=l > 0 

or 

01 > -a2fc j k=j~\ - -2 in 2 m -1.4 



IS 



20 



25 



which is not satisfied when $\ ——2. This suggests in- 
creasing the value of 0k by an amount greater than 
2— In2«0.6, say + 1, so that the new expression for z* 30 
from equation (8) is 

and ZA=y_i. Note that the addition of a constant equa- 
tion (8) has not changed the value of the derivatives 35 
thus ensuring that all three conditions are simulta- 
neously met. the net result is an upward shift by one 
unit of the transfer characteristics of FIG. 4 making all 
output values greater than zero. 

Another possible choice of weights and offsets could 40 
have linear uniform increment synaptic weights and 
quadratically scaled decoder offset values, as follows: 



1,2,3 ( ...,N 

»i = ok 
and 



45 



• , k = U 2, . . . . N. 



50 



Thus, the signal on the kin output tap of decoder 200 
would be representative of 



zt = ok 



55 



which satisfies the three conditions of expressions (5), 
(6) and (7) and is plotted in FIG. 5 with j as the parame- 60 
ter. In this case a = 1, but could assume any reasonable 
positive value that ensures non-saturating operation of 
the neuron's output non-linearity. A further advantage 
of the functional of equation (14) over that of equation 
(13) is the more restricted range of output variable, y*, 65 
that must be accommodated. Equation (14) is a qua- 
dratic rather than an exponential expression as in equa- 
tion (13). 



wjt-ln* 
«* = (*-e) 

so that 



1, 2 N 



z k =j\nk-{h- a) f 

where a is an arbitrary constant. As shown in FIG. 6, 
for a=2, equation (15) satisfies all three conditions. 

To those skilled in the art of neural nets and neural 
computing, other expressions satisfying the conditions 
in expressions (5), (6) and (7) may be found using the 
elementary mathematical tools invoked above. 

What is claimed is: 

1. A one-out-of-N neural network encoder and de- 
coder system comprising: 

(a) a set of N input terminals for designating one-out- 
of-N input states by applying a signal level to one 
terminal of the set of N input terminals; 

(b) encoding means connected to the set of N input 
terminals for producing, at an output terminal, a 
single multilevel output signal having N possible 
unique levels, each unique level corresponding to 
one of the designated one-out-of-N input states; and 

(c) docoding means with an input terminal coupled to 
the encoder means output terminal for receiving a 
signal representative of the level of the encoding 
means single output signal, the decoding means 
having N output terminals, each of the output ter- 
minals corresponding to one of the set of N input 
terminals connected to the encoding means, for 
decoding the received signal by generating, at the 
N output terminals, N signals such that the output 
terminal that corresponds to the designated input 
state has a value greater than any of the other N— 1 
output signal level values. 

2. A one-out-of-N neural network encoder and de- 
coder system comprising: 

(a) a set of N input terminals for designating one-out- 
of-N input states by applying a signal level to one 
terminal of the set of N input terminals: 

(b) encoding means comprising an N-input artificial 
neuron,, the artificial neuron N-inputs connected to 
the set of N input terminals for producing at the 
artificial neuron output terminal, a single multilevel 
output signal having N possible unique levels, each 
unique level corresponding to one of the desig- 
nated one-out-of-N input states; and - 

(c) decoding means comprising N single-input artific- 
ial neurons with each of the N single-inputs cou- 
pled to the encoding means output terminal for 
receiving a signal representative of the encoding 
means output signal level, each of the N single- 
input neurons having an offset input terminal and 
an output terminal, each of the output terminals 
corresponding to one of the set of N input terminals 

. connected to the encoding means for decoding the 
received signal by generating, at the N output ter- 
minals, N signal levels such that the output termi- 
nal that corresponds to the designated input state 
has a value greater than any of the other N— 1 
output signal level values. 
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3. The system of claim 2 wherein: the output of the k'* single-input artificial neuron a 

(a) the N-input artificial neuron of the encoding signal value z*. proportional to the value given by 
means has exponentially scaled uniform increment 

synaptic weights; u-w*'+0t 

(b) the N single-input artificial neurons of the decod- s , , 

ing means have linearly scaled uniform increment » mamnum decoder output value occurs 

synaptic weights; and f * c decoder output terminal k that corresponds 

(c) the offset bput terminals of the N single-input - ^JS%%g? £Z j> ^ 

. . r . . „ ~ . r . 7. The system of claim 3, further compnsinc an out- 

artificial neurons hive exponenhally scaled uni- put classifier means connected to the N output terminals 

form increment offset values applied. of thc decoding mcans , ^ output classifl( £ means ^ 

4. The system of claim 2 wherein: mg N output tcrminft i S| only onc output tcrminal 

(a) the N-input artificial neuron of the encoding active a t a time, the active output terminal being *e- 
means has linearly scaled uniform increment syrup- lected to correspond to the decoding means output 
tic weights; 15 terminal having the greatest output value, 

(b) the N single-input artificial neurons of the decod- 8. The system of claim 4, further comprising an out- 
ing means have linearly scaled uniform increment put classifier means connected to the N output terminals 
synaptic weights; and of the decoding means, the output classifier means hav- 

(c) the offset input terminals of the N single-input ing N output terminals, only one output terminal being 
artificial neurons have quadratically scaled uniform 20 active at a time, the active output terminal being se- 
increment offset values applied. lected to correspond to the decoding means output 

5. The system of claim 2 wherein: terminal having the greatest output value. 

(a) the N-input artificial neuron of the encoding 9 The system of claim 5, further comprising an out- 
means has linearly scaled uniform increment synap- P ut classifier means connected to the N output tenninals 
tic weights; 25 of the decoding means, the output classifier means hav- 

(b) the N single-input artificial neurons of the decod- N o^put terminals, only one output tenninal being 
ing means have logarithmically scaled uniform f a tUM ' °*P ut J'™"" 1 * 
increment synaptic weights; and "to* f. oorre ^ md <° U ? C 6 ^ in f mcans OUtpUt 

(c) the offset input tennbals of the N single-input te ™^ havin 8 the greatest output value. 

w : . y '™ , | j * r jo 10. The system of claim 7 wherein the output classi- 

amficial neurons have linearly scaled uniform in- ficr means j, a feed forward Haniming-net maximum- 

crement offset values applied. likelihood classifier. 

6 The system of claim 2 wherein: n The system of claim 8 wherein the output classi- 

(a) The N-input artificial neuron has a set of N synap- for means is a feedforward Hamming-net maximum- 
tic weights (vi, v 2 , . . . , v;, . . . , v N ) t one for each 35 likelihood classifier. 

of the N input states; 12. The system of claim 9 wherein the output classi- 

(b) the k' A neuron of the N single-input artificial neu- fier means is a feedforward Hamming-net maximum- 
rons has a synaptic weight value, w*, and an offset likelihood classifier. 

value, Ok, where k= 1, 2, ...» N, for producing at * * * * * 

40 
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